Combinatorial Penalties: Structure preserved by convex relaxations

In this paper, we study convex relaxations of combinatorial penalty functions. Specifically, we consider models penalized by the sum of an `p-norm and a set function defined over the support of the unknown parameter vector, which encodes prior knowledge on supports. We consider both homogeneous and non-homogeneous convex relaxations, and highlight the difference in the tightness of each relaxation through the notion of the lower combinatorial envelope of a set-function. We characterize necessary conditions under which the support of the unknown parameter vector can be correctly identified. We then propose a general adaptive estimator for convex monotone regularizers based on majorizationminimization, and identify sufficient conditions for support recovery in the asymptotic setting.